Multiple Kernel and Multi-label Learning for Image Categorization

MULTIPLE KERNEL AND MULTI-LABEL LEARNING FOR IMAGE CATEGORIZATION By Serhat Selçuk Bucak One crucial step in recovering useful information from large image collections is image categorization. The goal of image categorization is to find the relevant labels for a given image from a closed set of labels. Despite the huge interest and significant contributions by the research community, there remains much room for improvement in the image categorization task. In this dissertation, we develop efficient multiple kernel learning and multi-label learning algorithms with high prediction performance for image categorization. There are many image representation methods available in the literature. However, it is not possible to pick one as the best method for image categorization, since different representations work better in different scenarios. Multiple kernel learning (MKL), a natural extension of kernel methods for information fusion, is often used by researchers to improve image representation by integrating it to the learning step for selecting and combining different image features. MKL is mostly considered as a binary classification tool, and it is difficult to scale up MKL when the number of labels is large. We address this computational challenge by developing a stochastic approximation based framework for MKL that aims to learn a single kernel combination that benefits all classes. Another contribution of this dissertation is to develop efficient multi-label learning algorithms. Multi-label learning is arguably the most suitable formulation for the image categorization task. Many researchers have employed decomposition methods, particularly one-vs-all framework, with SVM (support vector machines) as a base classifier for addressing the image categorization problem. However, the decomposition methods have several shortcomings, such as their inability to exploit label correlations. Further, they suffer from imbalanced data distributions when the number of labels is large. Our contribution is to address multi-label learning via a ranking approach, termed multi-label ranking. Given a test image, multi-label ranking algorithms aim to order all the image classes such that the relevant classes are ranked higher than the irrelevant ones. The advantage of the proposed multi-label ranking approach, termed MLR-L1 (multi-label ranking with L1 norm), over other multi-label learning methods is its computational efficiency and high prediction performance. Image categorization is a supervised learning task, thus requiring a large set of training images annotated by humans. Unfortunately, labeling is an expensive process, and it is often the case that the annotators provide a limited set of labels, meaning that they only give a small subset of relevant tags for an image. One of the contributions of this dissertation is defining the problem of multi-label learning with incomplete class assignments and presenting a robust multi-label ranking algorithm, termed MLR-GL (multi-label ranking with group lasso norm), that addresses the challenge of learning from incompletely labeled data. Finally, we present a multiple kernel multi-label ranking algorithm to simultaneously address two essential factors for improving the performance of image categorization: Heterogeneous information fusion, and exploiting label correlations in multi-label data. We propose a multiple kernel multi-label ranking method that learns a shared sparse kernel combination that benefits all image classes. This way, we not only improve the training and prediction efficiency, but also improve the accuracy, particularly for classes with a small number of samples, by enabling information sharing between classes. We integrate the proposed MLR-L1 algorithm with an efficient semi-infinite linear programming (SILP) based MKL solver and develop a computationally efficient wrapper algorithm, termed MK-MLR (multiple kernel multi-label ranking).